• Download the zipfile from https://github.com/dlab-geo/geocoding

• unzip the files

• start RStudio

• start a new R script file

• set your working directory to the location of the unzipped files

About me…

Questions for you…

• Intro to Geocoding
• Review of Geocoding tools
• Practice Using a few tools
• Focus on ESRI ArcGIS Online Geocoder

Obtaining geographic coordinates for named places, street addresses, or zip codes.

Named places, codes or addresses

• country, state, city, building,
• street address or intersection,
• named landmarks like Mt Everest,
• zip codes

• Display locations on a map

• Link data by location

• Spatial analysis
  • Calculate distance, direction, area, etc.
  • Identify patterns & relationships:
    • clusters, outliers, neighbors

File of Input locations

• places of the same type in a standard format

A Geocoder

• Software and a reference database
• The software compares input locations to database & returns matches

Output file

• Geographic coordinates for each matched input location
• Metadata about the match

What types of data are available for geocoding

• Rooftop
• Parcel
• Street Addresses
• Dual Street Address Range
• Street Address Range
• Street, City, Zipcode, State

Where do you think the point for 275 5th St NE would be located?

Where do you think the point for 275 5th St NE would be located?

Addresses would match to centroid of parcel or rooftop

• Directions on your phone

• Search a place on Google Maps

• Consider your needs & options
• Choose geocoder
• Preprocess / clean your input data
• Geocode
• Evaluate your output
• Postprocess as needed

• Geographic & temporal scope
• Output type and quality
• Volume / speed
• Security
• Costs of software and data
• Ease of use

Many online tools will geocode "on the backend" but not give you access to the output.

• Google MyMaps

• Social Explorer

• Tableau

1. Open Sample Geocoding Data Google Sheet
2. Take a look at the SF Landmarks worksheet
3. Download SF Landmarks worksheet
   • File > Download as > CSV
4. Open Google MyMaps
5. Create a new Map
6. Click IMPORT and import the CSV file you just downloaded

I will just DEMO with Berkeley Grafitti worksheet data

When I have 1-10 locations to geocode…

• Google Maps

DEMO

When I have 10 - 1,000 US addresses & feel good about taxpayer supported tools…

• US Census Geocoding Tool

• Open the Census Geocoding website

• Select Find Locations Using...OPTION > Address Batch

• Download the sample CSV file from the Census website to see the needed file format

• View the Orig_Grafitti_street_addresses worksheet in the Sample Geocoding Data Google Sheet

• Copy & past the Grafitti to a new sheet and get it into the required format

• Download this revised worksheet to CSV

• Select the downloaded file to geocode the addresses using the Census geocoder

• Click Get Results and then review the output

When I have 1,000 - 10,000,000 and want a high level of control and output quality…

• ESRI
  • ArcGIS Online Platform
  • ArcGIS/ArcPro Desktop software

They are very good for geocoding and other geo tasks!

They are free to UCB Community via Campus Site License.

Pros

• No need to install software or data
• Access anywhere with calnet id & browser
• Simple geocoding workflow
• High quality geocoder, global in scope, always up to date
• Unlimited ESRI credits with the campus site license

Cons

• No customization of the geocoder
• No output metadata
• Complicated workflow to download geocoded data
• Can be slow with lots of data
• 5000 per person credit limit for AGOL
  • need to ask for more
  • One geocode “costs” 0.04 credits,
  • 125,000 geocodes = 5,000 credits

Pros

• Fast & Customizable
• Returns lots of metadata about the geocoded results
• Can use with online ESRI World Geocoding Service
• Can also use a local reference database
• Can be offline / secure

Available in the D-Lab

Cons

• Need to install software locally
• Only runs on Windows
• Can install & use local reference database but
  • it is huge (60-80GB),
  • needs to be updated and
  • requires a separate license file
  • only available for USA & Canada

• Download the New_Grafitti_street_addresses worksheet as CSV

• Login to cal.maps.arcgis.com

• Click on Sign in to University of CA Berkeley using CALNET Account.

• Go to Content

• Add Item > From My Computer

• You have the original uploaded CSV file
• and the GEOCODED Feature Layer (hosted)

• You have the original uploaded CSV file
• and the GEOCODED Feature Layer (hosted)

You need to export the GEOCODED feature layer to a CSV file and then download that CSV file to save the geocoded data locally.

Click on the feature layer to do this…

Now you can download

You can geocode 1- 50,000 addresses, place names or geocodes without problem in AGOL. If you have more than that you may want to think about your 5,000 ESRI credits.

You may need your credits for other work you do on AGOL.

You may need to allow for time to request and receive more credits for big geocoding jobs.

Use ArcGIS Desktop with a locally installed reference database if you have a huge number of addresses to geocode.

• Available on the D-Lab computers in 356 Barrows.

DEMO

Write a script (short computer program) to geocode your data.

• often written in R, Python, or Javascript

API: Application Programming Interface

• A set of functions, tools and protocols for making data requests using a computer program or script.

• US Census Geocoding API

• OpenStreetMap Nominatim Geocoding Service

• Mapquest Open

• Google Maps Geocoding API

• ESRI World Geocoding Service API

• And many others…

Prices and terms of use subject to change on freemium services!

View the OpenStreetMap Nominatim Geocoding Service Documentation

The format of a geocoding search request:

• https://nominatim.openstreetmap.org/search?q=

Try a geocoding search request in a web browser

https://nominatim.openstreetmap.org/search?q=Barrows Hall, Berkeley CA

Click on the details button in upper left when you get results

Almost all Geocoding APIs ask that you register for an API key

You use that key in your requests to use the service.

Some Freemium services ask for a credit card even if there is a free level of service.

Most folks use the geopy package because it provides a consistent interface to a number of different geocoding APIs.

DEMO if time using Google Collaboratory

More of a geocoding API specific landscape of packages

• ggmap package for the Google Geocoding API

• I have some R geocoding scripts on github you can review if you want to check them out.

Google Doc to guide you through geocoding with ESRI ArcGIS Online, Desktop or World Geocoding Service API

Geocoding Options with ESRI Software & Data

Many of the images used in this slidedeck were download from https://unsplash.com/, a great resource for beautiful free photos. See the R markdown file for photographer credits.